Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tools: add the bootstrapped _depot_tools dir to the PATH (for V8 tests) #38299

Closed

Conversation

DeeDeeG
Copy link
Contributor

@DeeDeeG DeeDeeG commented Apr 19, 2021

Short summary:

This script needs to execute a copy of ninja in order to complete.

The script downloads Google's depot_tools early on. (depot_tools includes a copy of the ninja binary.) I propose to add those to the PATH, so that running ninja later in the script succeeds for users at home running this outside of CI.

Background, context, more details:

The make-v8.sh script (used by the test-v8 Makefile target) implicitly requires a separate copy of ninja on the PATH, even though it bootstraps a copy of ninja for you (that it proceeds to not use).

This Pull Request adds the _depot_tools dir bootstrapped during this script to the PATH, before ninja is executed, taking advantage of the fact that _depot_tools conveniently includes a copy of ninja. This makes for one less system requirement to be able to finish the test-v8 Makefile target.

(I am not sure why this script chooses to add ~/_depot_tools to the PATH, as at least oiutside of CI on my personal machine, that directory never exists on my machine during the entire test-v8 Makefile target from beginning to end. But I interpret that as evidence that depot_tools is already meant to be on the PATH in some form or other.)

I was looking into documenting the system requirements needed to finish the test-v8 Makefile target. (I wanted to document that this is now the only part of this repository (that I know of) that still requires Python 2 after #36691, given that some Python scripts in the upstream depot_tools and v8 repositories still aren't Python 3 compatible).

When I was testing that Makefile target locally, I noticed something a bit strange (to my eyes) about an implicit ninja requirement:

  • The test-v8 Makefile target implicitly expects ninja on the PATH in order to complete
  • The test-v8 Makefile target ends up bootstrapping a copy of depot_tools before that, at deps/v8/_depot_tools
    • This includes a copy of ninja
  • A seemingly arbitrary path ~/_depot_tools is added to the PATH, but this isn't ensured to exist/isn't bootstrapped by the test-v8 Makefile target
    • (To my eyes, this path is arbitrary/maybe not useful, since it would be sheer coincidence if the user has depot_tools there, but I left it in case CI needs it... It should now be fully redundant as of this PR, so I think the script could probably skip adding that location to the PATH. Unless CI needs a custom depot_tools there with modifications? It works for me locally with upstream depot_tools, so I can't think of why that would be needed.)
  • The deps/v8/_depot_tools which was just bootstrapped is not put on the PATH
    • I feel like it should be? This gives us ninja, and we went out of our way to get it, so I think we should take advantage.

It feels like the bootstrapped _depot_tools is supposed to be on the PATH so we can use its copy of ninja
to complete the rest of building V8. Rather than documenting this quirky and redundant (maybe accidental) ninja requirement, I thought I'd have a go at making it unnecessary to manually ensure ninja is on the PATH, by using the copy that was just bootstrapped.

An alternative might be simply directly executing the deps/v8/_depot_tools/ninja binary without altering the PATH whatsoever. But I haven't yet tested whether that works. Works for me, not sure whether it works just as well in CI.

Ensures the `_depot_tools` dir bootstrapped during this script
is added to the PATH, before `ninja` is executed,
as `_depot_tools` already has a copy of `ninja` in it.
This means one less system requirement to be able to run this script.

(It seems like this was already intended to be on the PATH?)

Note: This script is used by the `test-v8` Makefile target.
@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. tools Issues and PRs related to the tools directory. v8 engine Issues and PRs related to the V8 dependency. labels Apr 19, 2021
tools/make-v8.sh Outdated Show resolved Hide resolved
@richardlau

This comment has been minimized.

Simplifies the script... Also finds _depot_tools relative to
the current path, rather than relative to the repo root.

This seems like a good idea, given that the rest of the script is also
relative to the current path (`deps/v8`), not the repo root.
@DeeDeeG
Copy link
Contributor Author

DeeDeeG commented Apr 23, 2021

I did some tiny updates:

  • Stopped having the script add ~/_depot_tools to the PATH, because this probably doesn't exist for most users. (As discussed above.)
  • Made the code finding _depot_tools simpler/smaller, and in the process, made the logic locating _depot_tools happen relative to the current dir deps/v8 rather than the repo root, to better match nearby lines of the script that are also relative to the current dir (deps/v8), not relative to the repo root.

(So, it's one less line of code, and slightly more robust to potential repo refactoring or moving folders around, or at least that's the idea.)

I opted not to remove depot_tools from PATH for those commands at the end, for two reasons:

  • I know CI does have depot_tools on the PATH, and I would like users running this outside of CI to get similar results as what will happen in CI.
  • V8 tells you to have depot_tools on the PATH in their official documentation for building V8

@richardlau
Copy link
Member

@jasnell
Copy link
Member

jasnell commented Apr 27, 2021

Landed in 6e3ce65

@jasnell jasnell closed this Apr 27, 2021
jasnell pushed a commit that referenced this pull request Apr 27, 2021
Ensures the `_depot_tools` dir bootstrapped during this script
is added to the PATH, before `ninja` is executed,
as `_depot_tools` already has a copy of `ninja` in it.
This means one less system requirement to be able to run this script.

(It seems like this was already intended to be on the PATH?)

Note: This script is used by the `test-v8` Makefile target.

PR-URL: #38299
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
targos pushed a commit that referenced this pull request Apr 29, 2021
Ensures the `_depot_tools` dir bootstrapped during this script
is added to the PATH, before `ninja` is executed,
as `_depot_tools` already has a copy of `ninja` in it.
This means one less system requirement to be able to run this script.

(It seems like this was already intended to be on the PATH?)

Note: This script is used by the `test-v8` Makefile target.

PR-URL: #38299
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
@targos targos mentioned this pull request May 3, 2021
targos pushed a commit that referenced this pull request May 30, 2021
Ensures the `_depot_tools` dir bootstrapped during this script
is added to the PATH, before `ninja` is executed,
as `_depot_tools` already has a copy of `ninja` in it.
This means one less system requirement to be able to run this script.

(It seems like this was already intended to be on the PATH?)

Note: This script is used by the `test-v8` Makefile target.

PR-URL: #38299
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
targos pushed a commit that referenced this pull request Jun 5, 2021
Ensures the `_depot_tools` dir bootstrapped during this script
is added to the PATH, before `ninja` is executed,
as `_depot_tools` already has a copy of `ninja` in it.
This means one less system requirement to be able to run this script.

(It seems like this was already intended to be on the PATH?)

Note: This script is used by the `test-v8` Makefile target.

PR-URL: #38299
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
targos pushed a commit that referenced this pull request Jun 5, 2021
Ensures the `_depot_tools` dir bootstrapped during this script
is added to the PATH, before `ninja` is executed,
as `_depot_tools` already has a copy of `ninja` in it.
This means one less system requirement to be able to run this script.

(It seems like this was already intended to be on the PATH?)

Note: This script is used by the `test-v8` Makefile target.

PR-URL: #38299
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
targos pushed a commit that referenced this pull request Jun 11, 2021
Ensures the `_depot_tools` dir bootstrapped during this script
is added to the PATH, before `ninja` is executed,
as `_depot_tools` already has a copy of `ninja` in it.
This means one less system requirement to be able to run this script.

(It seems like this was already intended to be on the PATH?)

Note: This script is used by the `test-v8` Makefile target.

PR-URL: #38299
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-ci PRs that need a full CI run. tools Issues and PRs related to the tools directory. v8 engine Issues and PRs related to the V8 dependency.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants